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ABSTRACT 

Medium-range  five-day  forecasts  from  the  U.S.  Navy 
Operational  Global  Atmospheric  Prediction  System  (NOGAPS) 
are  investigated  to  study  high  and  low  predictability 
periods  from  two  winter  seasons.  Northern  hemisphere  500  mb 
height  fields  are  scored  using  the  anomaly  correlation  coef- 
ficient- An  objective  method  is  used  to  choose  high  and  low 
scoring  periods  which  are  analyzed  using  height  tendencies 
and  wavenumber  structure.  Results  show  that  it  is  possible 
to  objectively  determine  why  some  high  and  low  periods 
occurred.  Flow   characteristics      leading      to    high      scoring 

five-day  forecasts  include:  long  wave  amplitude  decay,  tran- 
sition from  meridional  to  zonal  flow,  and  more  meridionally 
extensive  flow  patterns.  This  study  revealed  that  persis- 
tence is  not  a  good  indicator  of  model  performance,  and  no 
appreciable  skill  difference  exists  between  good  and  poor 
five-day  forecasts  at  the  48  hour  point-  However,  no  single 
measure  of  the  flow  patterns  is  found  to  be  a  unique  indi- 
cator  of  high    or  low    scoring    forecasts- 
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!•  INTRODDCTION 

Current  numerical  weather  prediction  systems  contain 
errors  from  various  sources;  data  inaccuracies,  initializa- 
tion, irregular  or  inadequate  distribution  of  observations, 
or  systematic  errors  from  the  model  itself  (Haltiner  and 
Williams,  1  980)  • 

Prediction  problems  arising  from  these  errors  can  be 
divided  into  three  categories,  depending  on  the  time  scale 
of  the  forecast  (Gronaas,  1982).  The  first  are  the  short 
range  forecasts,  mainly  affected  by  the  initial  conditions 
of  the  model.  The  second  are  the  long  -  term  forecasts 
which  are  largely  independent  of  the  initial  data-  This 
includes  the  problem  of  the  model  drifting  from  the  real 
atmospheric  climatology  to  its  own  climatology.  The  third 
category  between  these  extremes  is  the  medium  range  fore- 
cast. Bengtsson  and  Simmons  (1983)  define  this  time  scale 
as  being  a  few  days  to  a  week  or  two.  In  tnis  study  the 
medium  range  forecast  category  will  be  defined  as  roar  to 
five  days  in  length. 

Medium  range  forecasting  today  is  in  much  the  same  posi- 
tion as  short  range  forecasting  was  in  the  in  the  early  days 
of  numerical  weather  prediction  (NWP) .  Now,  as  then,  one 
finds  that  the  forecasts  have  some  useful  information  in 
them,  but  they  also  contain  a  considerable  amount  of  system- 
atic error  (Gronaas,  1932). 

Compared  to  the  early  NWP  models,  however,  the  system- 
atic errors  in  today's  complex  NWP  models  are  much  less 
predictable  (Gronaas,  1982).  The  end  result  is  tnat  the 
meteorologist  forecasting  in  the  medium  range  must  use 
subjective  synoptic  skills  to  extract  useful  information 
from  the  weather  forecasts  or  rely  on  climatology  and 
persistance  (usually  as  a  last  resort)  to  sort  out  the  poor 
from  tne  good  forecasts. 


It  is  known  that  there  are  times  when  the  medium  range 
forecasts  produced  by  any  of  the  major  operational  centers 
are  exceptionally  accurate,  and  other  times  when  they  are 
exceptionally  poor-  For  example,  time  series  representa- 
tions of  the  Anomaly  Correlation  Coefficient  (ACC)  in 
studies  by  Gronaas  (1982)  ,  Bengtsson  and  Simmons  (1983)  and 
Bengtsson  (1985)  at  the  European  Center  for  Medium  Range 
Forecasts  (ECMWF)  have  revealed  that  there  are  periods  when 
the  medium  range  forecasts  have  a  relatively  high  ACC  score. 
Assuming  that  values  of  this  correlation  coefficient  of  0.5 
or  0-6  indicate  the  limit  of  usability  of  a  forecast  field 
(e.g.,  Hollingswor th  et  a 1. ,  1980),  these  studies  show  there 
are  periods  of  time  when  the  0.7  value  is  exceeded  five  days 
beyond  forecast  time. 

These  "Spells  of  Time  when  tha  ACC  is  Relatively  High" 
(hereafter  referred  to  as  a  STARH  and  pronounced  'star') 
occur  without  any  apparent  periodicity  and  last  anywhere 
from  around  two  to  twenty-two  days  long,  Altaough  Bengtsson 
and  Simmons  (1983)  did  not  give  a  reason  for  STAHH  exis- 
tence, they  stated  that  investigating  the  reason  for  these 
spells  of  high  and  low  predictability  is  an  important 
research  topic.  Bengtsson  (1985)  suggests  that  these 
periods  might  occur  because  either  the  model  can  handle 
certain  situations  better  than  others,  or  that  the  accurate 
periods  occur  in  a  regime  that  is  inherently  more  predic- 
table. 

Gronaas  (1982)  showed  subjectively  that  these  periods 
are  to  a  large  extent  related  to  the  large  scale  flow 
patterns.  The  STARH  occurred  when  the  synoptic  wind  flow 
pattern  was  more  meridional  in  nature,  and  poor  forecast 
periods  occurred  during  times  when  zonal  flow  dominated. 
Nieminen  (1983)  also  indicates  that  the  ECMWF  model  has 
indications  of  being  more  skillful  in  periods  of  more  meri- 
dional flow,  for  example,  in  blocking  regimes. 
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Bettge  and  Baumhefner  (1984)  conclude  in  their  stirdy  of 
the  FGGE  data  that  model  forecast  skill  was  strongly  influ- 
enced by  the  planetary  wave  structure.  The  long-term  oscil- 
lations in  the  forecast  skill  were  found  to  be  related  to 
rapid  transitions  in  the  planetary  waves;  good  predictions 
occurred  when  the  waves  were  relatively  stationary  (blocking 
situations)  and  poor  forecasts  were  found  during  the  tran- 
sition periods.  These  poor  predictions,  or  "Forecasts  of 
Low    Predictability"   are   hereafter   referred   to    as    FLOPs- 

This  information  suggests  that  it  might  be  possible  to 
objectively  quantify  parameters  of  the  atmospheric  flow 
patterns  that  would  explain  the  cause  of  these  high  and  low 
predictability   periods     in   medium      range   forecasts-  It   is 

interesting  to  note  that  Elsberry,  et  al.  (1985)  indicate 
that  there  has  been  no  method  developed  or  tested  to  deter- 
mine the  accuracy  of  a  medium  range  forecast  based  on  the 
day  one  forecast  error.  Also,  Bengtsson  (1985)  stated  that 
the  cause  of  high  and  low  predictability  periods  is  a  funda- 
mental guestion  to  which  no  conclusive  answer  can  yet  be 
given. 

The  objectives  of  this  study,  then,  are  twofold.  First, 
to  develop  a  method  to  objectively  choose  high  (STARH)  and 
low  (FLOP)  predictability  periods.  Second,  to  investigate 
the  flow  patterns  of  the  chosen  periods  and  study  the  effect 
on   the    forecast    score    of    the    following: 

Persistence 

Early    error   growth 

Error    pattern    locations 

Average    height    field    tendencies      and   standard    deviation 
differences 

Zonal    wavenumber    structure 

Meridional    wavenumber   structure 

Chapter    two      of    this    study      discusses    the    the      source   of 

the    data   used    in   this      investigation    and   the   lethod    employed 

to   score   the    forecasts.       Chapter    three    details    the    objective 

method    for    choosing    STARH    aa d    FLOP      periods    and    a    summary    of 
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the  periods  selected.  Chapter  four  presents  .  the  analysis 
methods  employed  to  investigate  the  good  and  bad  forecast 
periods  and  a  summary  of  the  number  of  periods  accounted  for 
by  each  analysis  method.  The  conclusions  and  recommenda- 
tions for  future  research  are  summarized  in  Chapter  five. 
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II.  DATA  SOMMABI  415  SCORING  METHOD  EHPLOYED 

A.   DATA  SOUMABY 

Medium  range  forecasts  produced  by  NOGAPS  (Navy 
Operational  Global  Atmospheric  Prediction  System)  are 
studied  in  this  thesis.  The  data  was  retrieved  from  the 
Model  Output  Statistics  data_base  stored  at  the  Fleet 
Numerical  Oceanography  Center  (FNOC)  in  Monterey, 
California. 

NOGAPS  is  a  nine  layer  gridpoint  model  with  a  horizontal 
resolution  of  2.4°  latitude  by  3-0°  longitude.  The  maximum 
NOGAPS  forecast  length  is  five  days,  generated  once  a  day 
from  the  0000  GMI  analysis  data.  This  limits  the  study  to 
five-day  forecasts. 

The  current  version  of  NOGAPS,  2.1,  was  implemented  on  8 
December  1983.  Since  that  time,  an  error  in  the  latent 
heating  term  was  detected  (it  was  one  half  the  correct 
value).   This  error  was  corrected  in  April  1984. 

Only  the  northern  hemisphereic  section  of  the  global 
data  base  was  investigated.  This  is  because  forecast  accu- 
racy and  usefulness  is  substantially  less  in  the  southern 
hemisphere  due  to  a  sparsity  of  data.  This  was  demonstrated 
to  be  the  case  for  the  ECMWF  model  ty  Bengtsson  and  Simmons 
(1983),  and  was  also  found  to  be  true  of  the  NOGAPS  model  in 
preliminary  investigations  of  this  study. 

The  winter  season  forecasts  were  used  for  the  investiga- 
tion of  the  high  and  low  predictability  periods,  since 
winter  is  the  season  with  the  largest  number  and  the  most 
energetic  systems.  Also,  it  is  recognized  that  model  ACC 
scores  tend  to  be  higher  in  winter  than  in  summer. 

Tnere  were  only  two  winter  seasons  of  data  available 
since  the  NOGAPS  2.1  model  became  operational  on  8  December 
1983-  The  total  number  of  days  possible  to  use  in  this 
study  were  33      days  from  the  1983  winter   season  (8  December 
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1983  -  29  February  1984)  and  90  days  from  the  1984  winter 
season  (1  December  1984  -  28  February  1985).  The  actual 
number  of  days  available  for  use  in  this  study  was  a  few 
days  less  than  the  total  possible  due  to  missing  model  runs 
or  other  related  problems.  For  the  1983  winter  season,  80 
of  83  five-day  forecasts  (0-964  percent)  and  for  the  1984 
winter  season  82  of  90  five  day  forecasts  (0.911  percent) 
were  available. 

Time  constraints  limited  this  study  to  investigating 
only  one  level  of  the  atmosphere.  The  500  mb  level  was  used 
since  it  is  the  most  accurately  forecasted  level  in  the 
medium  range.  Also,  several  other  authors  (i.e., 
Hoilingswor t h  et  al. ,  1980,  Wallace  and  woessner,  1983, 
Gronaas,  1982,  Bettge,  1983,  Bengtsson  and  Simmons,  1983, 
Bettge  and  Baumhefner,  1984,  and  Bengtsson,  1985)  have  used 
the  500  mb  height  field  (alone  or  in  conjunction  with  the 
other  height  fields)  when  scoring  model  forecasts  or 
describing  the  state  of  the  atmosphere.  Investigating  this 
level  allows  for  comparability  between  tnis  and  other 
studies. 

B.   SCORING  TECHNIQOE 

The  anomaly  correlation  coefficient  (ACC)  is  used  for 
scoring  forecasts  in  this  study  following  Hollingsworth ,  et 
al. ,  (1980).  It  is  defined  as  the  correlation  between  the 
observed  and  predicted  deviations  from  climatology,  as  used 
by  Miyakoda  et  al.  (1972)  in  their  study  of  medium  range 
forecast  predictability. 

Gronaas  (1982)  states  that  the  ACC  is  probably  the  best 
single  scoring  technigue  available  when  used  with  care,  and 
he  notes  that  there  is  a  correlation  between  the  ACC  and  the 
standard  deviation  of  the  height  errors.  He  also  points  out 
that  the  ACC  is  not  the  ideal  scoring  method.  It  is  sensi- 
tive to  phase  errors  and  often  mesoscale  features  score  low 
due  to  errors  in  the  system's  propagation  speed.  Also, 
subjectively  graded  forecasts   may  score  high  even   when  the 
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ACC      produced    is      low.         For      example,      a      forecast    poor      in 
synoptic  detail   or   timing    may   score    badly   while   still   giving 
some    indication      of   an      overall  change      in   the      weather    type 
(Bengtsson    and   Simmons,    1983). 

Following  Miyakoda    (1972),    the    ACC   is   defined   to   be: 

2(F-C)*(0-C) 
ACC=  ^2.1) 


\/(2(F-C)2     *     2(0-C)2) 

where;  F   =    Forecast   parameter 

0  =    Observed   value 
C   =   Climatology    value 

Since   the   climatology    data  base   was    not   available   from    FNOC, 
a   monthly    mean    500    mb    height    was    used    for   climatology. 

A  further  gualif ication  was  made  as  to  the  latitudes 
over  which  the  ACC  was  calculated,  following  Bengtsson  and 
Simmons  (1983).  Only  those  grid  points  within  the  latitude 
band  20°  to  82.5°  were  included  in  the  calculation  to  avoid 
the  tropics  and  the  polar  region.  This  is  done  because  all 
present  day  NWP  models  are  poor  in  the  tropics  and  finite 
differencing  type  models  require  special  handling  at  the 
poles    (Haitiner   and   Williams,    1980). 
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III.    DISCUSSION    OF   SELECTED    PERIODS 

A.       OBJECTIVE   DEFINITION    OF    PEEDICTABIIITY 

An  objective  measure  of  what  constitutes  a  high  or  low 
predictability  period  in  a  numerical  weather  prediction 
forecast  is  desired.  Hollingsworth  et  al.  (1980),  states 
that  an  anomaly  correlation  coefficient  valus  of  0.5  or  0.6 
is   the    limit    of    useful   predictability. 

Gronaas  (1982)  divides  the  high  versus  low  predict- 
ability forecasts  by  comparing  the  average  3-,  5-,  and  7-day 
ACC  values  for  a  period  of  interest  (the  periods  being  from 
3  to  26  days  long)  to  the  same  for  the  yearly  averaged 
values,  and  he  subjectively  decides  if  each  one  is  a  high  or 
low    predictability    period. 

This  study  will  only  examine  the  most  significant 
periods  of  the  five-day  forecasts  that  were  well  or  poorly 
forecasted..  To  this  end  the  high  and  low  predictability 
periods    are    chosen    as    follows: 

•  The  ACC  is  calculated  in  the  latitude  band  from  20°  to 
82.5°    N    for   the   entire    winter. 

•  The  good  periods  are  initially  determined  by  those 
periods  where  the  kCC  is  above  0.6  (the  limit  ot  useful 
skill).  and  the  poorly  forecasted  periods  being  those 
below  0,4  (to  limit  the  data  set  to  the  very  worst 
cases)  . 

•  The  ACC  for  these  periods  are  averaged,  and  compared 
to  the  mean  and  standard  deviation  of  the  ACC  for  the 
entire    winter    data   set. 

•  If  tae  average  ACC  for  the  period  is  one  or  more  stan- 
dard deviations  above  or  below  the  long  term  mean,  the 
period  is  accepted  as  either  a  STARH  (nigh  score)  or  a 
FLOP    (low    score) - 

B-       SUMMARY    OF    SELECTED    PERIODS 

Figs.       A.I    and      A. 2   show    the    time    series      of    the    anomaly 

correlation    coefficient      plotted    as      a    function      of    the      day 

(solid    line)  for    both    winter      data    sets.  The   convention 

adopted    for       these    graphs    is    that      the    score    listed      above    a 

day      is    the      ACC   score      of      the   five-day      forecast    that      was 
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generated  on  that  day  (and  whose  verification  time  was  1 20  h 
later) .  also  shown  on  these  figures  is  the  persistence  ACC 
score,  plotted  as  a  function  of  the  day  (dashed  line).  This 
score,  computed  as  was  the  five-day  forecast  ACC,  uses  the 
analysis  field   as   the    five-day  forecast. 

Table  1  is  a  summary  of  the  STARH  and  FLOP  periods 
selected.  The  statistics  listed  in  Table  1  show  each  period 
meets  the  selection  criterion  established  in  the  previous 
section.  Shown        are      the        thirteen      selected        periods, 

consisting  of  six  STAEH  and  seven  FLOP  periods  evenly 
divided  between  the  two  winters.  The  rationale  for  the 
numbers  assigned  to  the  periods  is  that  the  one-digit 
numbers  are  periods  from  the  1983-84  winter  season,  while 
the  two-digit  numbers  refer  to  periods  from  the  1984-85 
season.  Winter      1983-84      periods      are      not      consecutively 

numbered  because  other  periods  had  been  initially  selected 
but  were  later  discarded  when  the  selection  criterion  was 
imposed . 

The  period  cf  time  over  which  the  STARHs  or  FLOPs  exist 
varies  from  the  shortest  of  two  days  (periods  S5,  F4,  and 
S12)  to  the  longest  of  twelve  days  (period  S10).  It  is  also 
interesting  to  note  that  like  periods  do  not  all  occur  in 
the  same  month  or  in  a  close  period  of  time,  but  exist  in 
more   of    a   random   distribution    pattern. 

These  selected  periods  are  now  studied  in  detail  to 
better  understand  the  reasons  for  such  variation  in  medium- 
range   forecast   performance. 
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IV,  INVESTIGATIOHS  OH  BEDIOM  RANGE  FORECASTING 

The  flow  patterns  of  the  chosen  periods  are  studied  in 
order  to  quantify  the  forecast  score  variance  due  to  persis- 
tence, early  error  growth,  error  pattern  locations,  average 
height  field  tendencies,  standard  deviation  differences, 
zonal  wavenumber  structure  and  meridional  wavenumber 
structure, 

A.   THE  ROLE  OF  PERSISTENCE 

Persistence  of  the  atmospheric  flow  patterns  is  natu- 
rally a  candidate  to  explain  the  existence  of  high  and  low 
predictability  periods.  If  persistence  is  a  strong  factor, 
one  would  expect  that  a  flow  pattern  that  exhibited  little 
change  through  the  forecast  period  would  verify  with  a  high 
score,  while  periods  of  strong  flow  pattern  change  would 
verify  poorly- 
Figs-  A.I  and  A. 2  show  the  persistence  score  as  a  func- 
tion or  forecast  day.  Examining  these  figures  for  the  STARH 
and  FLOP  periods,  one  sees  that  persistence  is  a  poor  indi- 
cator of  model  performance.  For  example.  Fig,  A-1  shows 
persistence  ACC  score  to  peak  at  the  same  time  as  STASH  SI 
(26-30  December  1983),  but  it  also  peaks  at  the  same  time  as 
FLOPS  F1  (30-31  January  1984)  and  F8  (15-21  February  1984). 

The  number  of  selected  periods  explained  well  or  poorly 
by  the  persistence  method  is  listed  in  Table  2  One  finds 
that  five  periods  can  be  accounted  for  Ly  the  persistence 
method  (SI,  F4,  F7,  S11,  and  F12) ,  but  six  others  (S2,  S5, 
F1,  F8,  S12,  and  F11)  are  periods  where  the  persistence 
value  is  just  opposite  to  what  would  be  expected  using  this 
analysis  method.  It  is  evident  that  with  as  many  periods 
occurring  opposite  of   what  is  expected  in   both  winters  and 
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for  both  STARHs  and  FLOPs,  persistence  is  not  a  good  indi- 
cator  of  model   forecast   score   in    the    medium-range- 

Additional  tests  were  run  on  the  correlation  between  the 
model  forecast  ACC  time  series  curve  and  the  persistence  ACC 
curve  of  Figs.  A.I  and  A. 2  The  results  are  shown  in  Table  3 
for  winter  1983-84  and  in  Table  4  for  winter  1984-85.  If 
persistence  was  to  be  a  good  indicator  of  the  forecast  ACC, 
the  correlation  between  the  model  and  persistence  forecast 
ACC  curves  would  be  high  (close  to  one).  Any  correlation 
would  produce  a  value  at  least  above  zero-  It  can  be  seen 
in  these  tables  that  only  for  one  month  (December  1983)  is 
the  persistence  ACC  curve  highly  correlated  with  the  fore- 
cast ACC  curve  (correlation  value  of  0-47) .  This  correla- 
tion is  also  evident  on  Fig-  A.I  Three  of  the  months  show 
negative  correlation  values  (January  and  February  1984,  and 
February  1985),  while  the  other  two  months  have  small  posi- 
tive values. 

It  is  interesting,  that  in  one  month  (December  1983)  of 
the  six  months  studied,  the  medium-range  forecast  score 
variance  was  so  highly  correlated  with  persistence.  Quiroz 
(1984)  states  that  December  1983  was  a  month  of  unusually 
strong  blocking  patterns  over  North  America.  This  may,  in 
part,  explain  wny  the  December  ACC  score  so  strongly  paral- 
leled the   persistance    score. 

In  any  case,  the  fact  that  persistence  is  not  a  good 
indicator  of  forecast  skill  in  this  data  set  is  consistent 
with  the  results  of  Bengtsson  and  Simmons  (1933)  who  also 
found  high  medium-range  forecast  scores  in  times  when  a 
large  changes  occurred  in  the  haxght  fields  through  the 
forecast  period. 

B-       THE    EOLE    OF    EARLY    EBfiOR    GBOHTH 

More  data  available  for  a  NWP  model  initialization 
should  lead  to  better  short  and  medium-range  forecasts- 
This  suggests  that  one  reason  for  poor  model  performance  in 
the    medium-range      might   be      the    quality      or    quantity      of    the 
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initial  data.  If  model  skill  is  affected  by  these  errors  in 
the  initial  data  field,  one  would  expect  to  see  a  lower  ACC 
score  immediately  in  the  forecast  period  for  the  FLOP 
periods,  and  a  higher  ACC  score  in  this  same  period  for  the 
STARHs. 

The  average  ACC  values  of  each  selected  period  as  a 
function  of  time  are  presented  in  Figs.  A-3,  A-4,  A. 5,  and 
A. 6  -  These  figures  show  the  trend  of  the  average  forecast 
score  for  each  period  over  the  five-day  forecast  period. 
Comparing  the  STARH  periods  of  winter  83-84  on  Fig.  A- 3  to 
the  FLOPS  of  that  winter  on  Fig.  A. 4,  little  difference 
between  the  STARHs  and  FLOPs  is  present  out  to  48  hours. 
Only  one  FLOP,  F8,  is  seen  to  have  a  noticeably  lower  ACC  at 
the  48- hour  point.  F4  and  F7,  hDwever,  cannot  be  distin- 
guished from  the  STARHs  at  this  time,  and  F1  is  only 
slightly  lower  than  the  the  lowest  scoring  STARH  at  48  hours 
(S2)- 

It  is  interesting  to  note  that  F8,  although  scoring  much 
lower  than  the  other  FLOPs  at  24,  43  and  72  nours,  is  at 
almost  the  same  ACC  value  at  96  and  120  nours.  This  indi- 
cates that  early  errors  for  F8  did  not  produce  a  120-hour 
ACC    score    significantly    below    the    other   FLOPs. 

Comparing  the  STARHs  and  FLOPs  from  winter  84-85  on 
Figs.  A-5  and  A. 6  shows  similar  results.  At  48  hours  the 
ACC  of  the  STARHs  are  all  higher  than  the  FLOPs,  but  the 
STARHs  and  FLCPs  are  very  close  on  the  ACC  scale.  They  are 
all  within  0.05  of  each  other,  and  the  highest  FLOP  is  only 
about   0.02    below    the    lowest      STARH.  Although    there   are    not 

enough  data  to  assign  a  statistical  significance  to  this 
grouping  of  points,  it  is  clear  that  for  both  winters  the 
largest  difference  in  ACC  between  the  STARH  and  FLOP  periods 
lies  at  the  72-hour  point  and  beyond.  In  the  summary  of 
results    (Table    2)       only    one    of   the    13    periods    (F8)       shows   an 
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expected  result.  The  rest    show      no   indication      that    early 

error   growth    was  an  indicator   of   the   five-day   ACC   scores. 

The  obvious  conclusion  is  that  the  errors  present  early 
in  the  forecast  period  of  the  FLOP  periods  are  not  signifi- 
cantly different  from  those  present  in  the  STARH  periods, 
and  that  initial  data  errors,  as  ravealed  by  the  ACC  scoring 
method,  do  not  seem  to  play  a  major  role  in  determining 
forecast  skill  in   the    medium-range.  This   result   is   depen- 

dent     upon    our      choice      of      scoring    tae     forecasts-  Other 

medium-range  skill  scores  may  be  more  sensitive  to  early 
errors. 

C.       THE    ROLE    OF    EEEOfi    PATTERNS 

1.       Summary      of    Error      Sources    for      the      STARH   and      FLOP 
Periods  " 

This  section  summarizes  the  features  in  the  analysis 
charts  of  the  500  mb  height  field  that  differed  from  the 
forecasted  height  field  for  all  the  good  and  poor  forecast 
periods.  By  comparing  the  error  patterns  (generated  by  the 
subtracting  the  500  mb  verifying  day  analysis  from  the  500 
mb  five-day  forecast)  to  the  verifying  day  analysis,  one 
could  identify  the  flow  pattern  features  related  to  the 
error  patterns.  Retracing    in      time    the      evolution   of      the 

feature,  it  was  possible  to  gain  some  insight  as  to  why  some 
periods   scored    poorly    and    why    others   scored    rather    well. 

Examining  all  the  selected  periods  in  the  winter 
season  1983-84  in  this  fashion  revealed  that  tne  major 
sources  of  errors  in  the  forecast  field  could  be  reduced  to 
a  few      features    in      common   to   all      the    periods.  While   tne 

basic  cause  of  the  errors  was  found  to  be  the  same  for  both 
the  STARHs  and  FLOPs,  it  should  understood  that  the  sizes 
and  amplitudes  of  systems  causing  the  error  patterns  in  the 
STARHs  were  much  smaller.  The  most  common  features  that 
caused    the    large   error    patterns   are    listed    below: 

•  Formation    cf   a   major   ridge      late    in   the    forecast    period 
(around    day   three   to   day    five)  - 

•  The  formation      of    a    major      trough    late    in      the    forecast 
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period     (around   day   three   to   day   five). 

•  The  rapid  deepening  of  a  Low  pressure  center  or  devel- 
opment of  a  shortwave  trough  within  days  three  to  five 
ox   the    forecast    period- 

•  The  formation  of  a  cutoff  low  pressure  center  after  day 
two  of    the   forecast    period. 

•  The  regression  of  a  low  or  high  pressure  center  after 
day  two    of   the   forecast    period. 

•  Inability  to  maintain  the  intensity  of  major  high  or 
low  pressure  centers  that  persist  throughout  the  fore- 
cast  period. 

These  features  were  almost  always  under  forecasted 
by  NOGAPS.  That  is,  the  highs  were  too  low  and  the  lows 
were      too      high      in      the      five-day      forecast-  There      were 

instances  in  which  the  sense  of  the  error  was  just  the  oppo- 
site as  this,  but  these  cases  were  by  far  fewer  and 
accounted   for    a   only    a   small    part   of   the   total   error   field. 

It  is  interesting  to  note  that  in  the  summary  of  the 
errors  listed  previously,  most  of  the  error  causing  systems 
were  ones  that  developed  after  day  two  in  the  forecast 
period.  This  corresponds  with  Figs.  A-3  through  A. 6  which 
show  the  largest  difference  in  ACZ  scores  between  the  STAEH 
and  FLOP  periods  is  after  day  two  as  well. 
2»      Geographic  Location    of  Errors 

An  example  of  the  types  of  errors  encountered  is 
shown  on  Figs.  A. 7  through  A. 11  .  Fig.  A. 7  is  a  northern 
hemisphere    polar   stereographic    (PS)  500   mb    height   analysis 

from  23  December  1983.  The  five-day  forecast  made  from  this 
analysis  was  a  FLOP  (F1)  and  verified  against  the  500  mb 
height  field  analysis  of  28  December  1983,  shown  in  Fig.  A.  9 
The  forecast  errors  (forecast  minus  observed  height  values) 
are  shown  on  Fig.  A. 8,  with  contour  intervals  of  160  meters. 
Comparing  this  with  Fig.  A. 9  one  can  quickly  discern  that 
the  largest  area  and  amplitude  errors  are  associated  with 
features  that  intensified  or  developed  within  the  forecast 
period.  The  most  noticeable  features  on  Fig.  A. 9  associated 
with  large  error  patterns  are  the  omega  block  over  western 
Europe,       the    family    of   lows      across    the    North    Pacific   Ocean, 
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the  ridge  along  the  west  coast  of  North  America,  and  the 
trough  over  north  central  Canada  (the  area  around  Baffin 
Island)  . 

The  five-day  forecast  generated  from  the  28  December 
analysis  was  a  STARH,  and  verified  against  the  analysis  of 
02  January  1984,  shown  on  Fig.  A-  11  The  plot  of  the  error 
patterns  for  this  forecast,  similar  to  Fig-  A. 8,  is  shown  on 
Fig.  A. 10  Not  surprisingly,  the  size  of  and  number  of  the 
error  patterns  greater  that  160  meters  is  much  smaller, 
indicating  the  forecast  field  captured  most  of  the  changes 
that    had  occurred- 

The  features  on  Fig.  A. 11  that  correspond  to  the 
error  patterns  of  Fig-  A.  10  are  not  nearly  as  spectacular  as 
they  were  for  Fig.  A.  8  Two  of  the  error  patterns  (over 
England  and  in  the  mid-Pacific  Ocean)  are  associated  with 
mostly  zonal  flow, while  two  others  (Greenland-Newfoundland 
area  and  the  Black  Sea  region)  are  associated  with  short 
wave  troughs-  The  remaining  error  pattern  is  related  to  a 
diffluent   trough  along   the   west   coast    of   North   America- 

In  looking  at  all  the  error  pattern  maps  for  ail  the 
STARH  and  FLOE  periods  of  the  83-84  winter,  it  was  evident 
that  most  of  the  differences  in  error  patterns  between  the 
STARH  and  FLOP  periods  are  within  the  latitude  band  of 
roughly  30°  to  60°.  Both  STARH  and  FLOP  periods  were  seen 
to  have  large  errors  north  of  60°  north  latitude,  but  not 
too  surprisingly,  only  the  FLOP  periods  had  large  area  and 
amplitude  errors  in  several  locations  within  the  30°  to  b0° 
latitude  band- 
Both  STARHs  and  FLD Ps  had  errors  caused  mostly  by 
systems  that  developed  late  in  the  forecast  period  (after 
day  two).  The  FLOP  periods  had  large  scale  development  of 
systems  roughly  witnin  this  latitude  band  over  the  forecast 
period,  while  the  STARHs  did  not  show  this  feature-  To 
determine  if  these  developing  large  amplitude  error  patterns 
were   occurring      systematically  in    the    same      geographic    loca- 
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tions,  the  error  patterns  were  averaged  within  the  latitude 
band  30°  to  60°^  and  each  day's  values  were  plotted  as  a 
function  of  longitude  in  a  Hovmiiller  diagram  shown  on  Fig. 
A. 12   for  winter    1983-84,    and    Fig.    A- 13   for    winter    1984-85- 

These  two  figures  show  the  mean  error  at  each  longi- 
tude through  time,  with  time  increasing  to  the  top  of  the 
graph-  Negative  values  of  the  forecast  minus  observed  field 
are  shown  as  dashed  lines- 
Cross  referencing  the  days  when  the  FLOPs  occurred 
with  the  longitude  of  the  large  error  areas  shows  that  for 
both  winters  no  one  longitude  has  positive  or  negative 
errors  that  uniguely  define  FLOPs.  The  error  patterns  have 
rather  a  random  appearance  to  them  without  much  longitudinal 
alignment. 

It  is  well  known  that  certain  areas  of  the  world 
experience  more  freguent  cyclogenesis  than  others  (lee  of 
major  mountain  ranges,  east  coasts  of  continents,  etc.,). 
What  Figs.  A. 12  and  A- 13  indicate  is  that  the  FLOPs  are 
associated  with  large  scale  development  (not  just  cyclo- 
genesis) almost  anywhere  within  the  latitude  band  30°  to  60° 
N-  Apparently,  no  one  area  (longitude)  is  preferred  over 
another   in    the   generation   of    a   poor    forecast- 

D-       ROLE   OF    HEIGHT    CHANGES 

Blocking  situations  have  been  attributed  to  hign 
predictability  periods  by  some  authors  in  past  literature 
(Gronaas,  1982,  and  Bettge  and  Baumhefner,  1984).  Fig.  A.  9, 
the  analysis  height  field  for  28  December  1983  (discussed 
earlier),    shows    such    a    case.  Here,    the   500    mb    flow    pattern 

is   in      a  low      index   state    (strong      meridional    flow)  with    a 

classic  omega  block  over  western  Europe  and  a  simple  block 
over  the  Gulf  of  Alaska- 
It  was  stated  earlier  that  the  main  difference  between 
the  STARH  and  FLOP  periods  is  the  lack  of  development  of 
systems  over  the  five-day  forecast  time  for  the  STARHs.  The 
analysis  charts      from   28    December      1983    (Fig.       A-9)  to   the 

24 


verifying   day   of   02   January    1984    (Fig.         A. 11)    and    the    error 
pattern   associated      with    the    five-day    forecast       (Fig.      A- 10) 
presents  an    excellent   example   of   this   difference-      The    large 
blocking  systems     of    28      December   decayed      and    were      minimal 
sources   of    error  while   no   new   systems    developed. 

The  tendency  of  the  STARH  periods  to  exhibit  less  devel- 
opment and  more  of  a  decay  of  the  existing  high  amplitude 
systems  can  be  described  as  the  flow  shifting  from  a  low  to 
a  high  index  regime  (meridional  flow  to  zonal  flow  tran- 
sition) -      The   opposite    would    then   apply   to   the    FLOPs. 

NOGAPS  systematic  errors  at  500  mb  (Boyle  and  Wash, 
1985)  are  to  fill  troughs  and  weaken  ridges.  Thus,  NOGAPS 
will  verify  better  in  the  medium-range  when  the  atmosphere 
trend  (relaxing  of  high  amplitude  systems)  follows  the  model 
systematic    error   trend. 

To  objectively  quantify  this  flow  pattern  behavior,  the 
500  mb  analysis  height  fields  were  averaged  between  30°  and 
60°  N  latitude  (the  latitude  band  in  which  it  appeared  the 
STARHs  had  less  errors  and  the  FLOPs  had  more  errors).  Each 
day's  average  values  were  plotted  as  a  function  of  longitude 
in  a  Hovmiiller  diagram  as  shown  in  Figs.  A. 11  and  A.  15  . 
The  zonal  mean  is  removed  so  the  low  height  areas  show 
dashed  lines,  the  heavy  line  is  the  zero  contour,  and  the 
high  height  areas  show  as  solid  lines.  The  heights  were 
contoured   at    an   interval    of    80  meters. 

These  diagrams  show  the  height  field  anomaly  at  each 
longitude  for  each  day.  Time  increases  towards  the  top  of 
the  graph,  which  allows  one  to  discern  how  the  anomalies 
(the  highs  and  lows)  change  over  time  with  respect  to  any 
one  location.  The  figures  show  the  typical  features  of  the 
winter  500  mb  patterns,  the  long  wave  troughs  at  longitudes 
150°E    and  70°W,    with   ridges    at   70°E,    130°W   and    20OK. 
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If  the  proposed  mechanisms  are  correct,  one  would  expect 
that  as  you  move  forward  in  time  in  a  STARH  period  from  the 
day  a  forecast  was  generated  to  its  verifying  time,  the 
highs  would  be  decaying  and  the  lows  would  be  filling-  It 
is  pleasing  to  note  that  this  is  exactly  what  is  seen  for 
some  of  the  periods.  In  Fig.  A.  14  ,  for  winter  1983-84, 
this   is   the      case   for    periods   SI    and   S 5.  By    examining    the 

height  field  for  a  day  in  these  periods,  and  then  glancing 
forward  in  time  (towards  the  top  of  the  figure)  ,  one  can  see 
that  over  the  five-day  forecast  period,  the  lows  are  filling 
and  the  highs  are  decaying  for  the  most  part-  The  same  is 
true  of  periods  S10  (the  last  half) ,  and  S12  on  Fig-  A- 15  . 
The  shifts  in  height  anomalies  expected  for  the  FLOP  periods 
are  just  the  opposite  as  those  for  a  STARH  (i.e-,  the  lows 
should  deepen  and  the  highs  should  build  over  the  forecast 
period)  -  This  can  be  seen  for  F4  and  F7  in  Fig-  A- 1 4  and 
for    F10,   F11    and   F12    on    Fig-     A. 15 

Unfortunately,  the  height  anomaly  trend  for  each  type  of 
period  does  not  always  occur  as  expected.  The  first  part  of 
S10  on  Fig.  A-15  is  the  best  counter  example.  The  highs  are 
building  and  the  lows  are  deepening,  but  the  model  scored 
high  using  the  ACC  (this  is  certainly  what  the  numerical 
modelers  like  to  see)-  These  are  periods  when  the  model  can 
successfully  predict  wave  amplification  in  the  medium- range. 
Also,  using  shifts  in  height  anomalies  over  time  as  a  method 
of  determining  model  performance  for  the  medium-range  fore- 
casts does  net  uniguely  define  STARH  and  FLOP  periods. 
There  are  several  periods  that  show  the  tendencies  for  a 
STARH  (FLOP)  period,  yet  did  not  score  significantly  high 
(low)  with  the  ACC.  For  example,  on  Fig-  A- 14  one  sucn 
STARH-like  period  is  from  about  8  to  10  February  1934,  and  a 
FLOP- like  period  is  around  2  to  3  January  1984.  This  means 
that  although  the  technique  works  for  many  of  the  STAEHs  and 
FLOPS,     it    is    not    unique    solely   to    those    periods. 
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To  better  illustrate  the  shifts  in  the  height  anomalies 
over  time,  the  standard  deviation  of  the  mean  height  is 
calculated  for  the  height  field  represented  on  Figs.  A.  14 
and  A. 1 5  .  The  standard  deviation  of  each  day's  height 
field  is  shown  on  Fig.  A. 16  for  winter  83-84  and  Fig,  A, 17 
for   winter    84-85. 

It  is  expected  that  a  low  iadex  period  would  have  a 
large  standard  deviation,  as  compared  to  a  hign  index  period 
where  a  smaller  standard  deviation  value  would  be  calcu- 
lated. Detecting  the  change  in  tha  standard  deviation  value 
over  the  five-day  forecast  period  is  simplified  by  using 
Figs.  A. 18  and  A- 19  (standard  deviation  difference  graphs 
for      83-84      and      84-85).  This      graph     is      constructed      by 

subtracting  the  standard  deviation  of  the  day  at  the  veri- 
fying time  from  the  value  at  the  analysis  time.  The  differ- 
ence is  plotted  as  a  function  of  time  when  the  five-day 
forecast  was    generated- 

It  is  expected  that  the  STAEHs  would  be  indicated  as 
positive  peaks  (showing  a  tendency  of  moving  from  a  low  to  a 
high  index  regime)  and  FLOPS  as  negative  minima.  This  anal- 
ysis technigue  has  some  successes  and  some  interesting 
results  (results  occurring  opposite  to  expectations  are 
termed      "interesting"      vice      "bad"        or      "failures").  For 

example,  in  Fig,  A. 19  (winter  1984-85),  the  two  largest 
positive  peaks  precisely  define  two  STARH  periods  (SI  2  ani 
S10,  as  indicated).  Analogously,  F10  (a  FLOP)  is  precisely 
defined  as  a  large  "valley"  in  the  negative  portion  of  the 
graph  as  expected.  This  indicates  that  the  shift  in  height 
anomalies  detected  on  Figs.  A- 14  and  A. 15  can  be  objec- 
tively quantified  and  is  successful  in  describing  some 
periods  , 

The  other  three  periods  of  this  winter,  however,  are  not 
quite  as  successful.  The  standard  deviation  difference  for 
STARH    (S11)     is    generally    positive,       but   not   exceptional     (the 
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period  21-25  December  which  did  not  have  a  spectacularly 
high  or  low  ACC  was  generally  more  positive  than  the  period 
27-31  December  of  S11).  Similarly,  the  standard  deviation 
differences  of  periods  F11  and  F12  are  both  mostly  negative 
(as  expected),  but  they  are  not  within  distinctly  low 
negative  areas  (the  period  9-12  February  is  a  distinct 
minimum,  but  is  not  a  STAEH  or  a  FLOP  according  to  the  ACC 
scores)  - 

The   most    interesting    part   of   Fig.    A- 19   is   the   first    part 

of   S10      that    lies  in    a      large   negative    portion    of      the    graph 

(22-29    January).         This   region   has   every   indication    of    being 

a    poorly     predicted    period,         yet   the      model    forecasted      the 

changes   that    occurred    very   well- 

The  successes  are  fewer  for  the  1983-84  winter  period 
(Fig.  A. 18).  For  example,  S5  and  SI  are  within  positive 
peaks  of  the  graph,  but  other  areas  of  higher  positive 
values  exist  (15-16  December,  6-10  January,  and  17-18 
January)  that  were  not  spectacularly  high  or  low  ACC 
periods.  Also,  the  FLOPS  F1,  F4,  and  F7  do  not  lie  exactly 
in  a      local    minimum      portion    of      the   graph.  Nevertheless, 

these  FLOPS  are  almost  all  on  the  negative  side  of  the 
graph.  Neither  does  STARH  S2  lie  in  a  peak  of  the  standard 
deviation  values,  and  the  last  day  of  the  period  (21  January 
1984)    is  in    the    negative    portion    of    the   graph- 

The  two  most  interesting  features  of  this  winter  are 
that  the  large  minimum  at  22-23  January  does  not  delineate  a 
FLOP,  and  that  F8  has  standard  deviation  differences 
entirely  on  the  positive  side  of  the  graph.  This  means  that 
the  model  did  poorly  even  though  the  flow  regime  was 
changing  to  a  more  zonal  flow  pattern  (expected  to  produce 
good   forecasts)  . 

For  both  winters,  these  graphs  also  serve  to  show 
periods  wnen  the  ACC  was  neither  spectacularly  high  or  low, 
yet    look   like    STARHs    or   FLOPs      because    of   the    large    positive 
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or     negative      standard   deviation      differences.  This      more 

clearly  illustrates  that  this  height  anomaly  tendency  is  not 
a   unigue  indicator    of   a   STARH    or   FLOP    period. 

The  summary  of  which  periods  were  not  described  well  by 
this  method  is  shown  on  Table  2  In  all,  four  periods  had 
expected  results  (SI,  S5,  S12,  and  F  10),  one  had  results 
opposite  to  expectations  (F8),  and  the  remaining  eight 
periods  had  results  that  were  generally  as  expected,  but 
were  either  not  spectacular  or  had  one  or  more  forecasts  in 
the    period    that    did   not   fit    the   hypothesis. 

£.       ZONAL    WAV£NO£iBEfi    STEOCTORE 

Bettge  and  Baumhefner  (1984)  used  zonal  wavenumbers  one 
and  two  in  describing  the  systematic  errors  in  NMC  (National 
Meteorological  Center)  forecasts.  They  conclude  that  the 
power  (amplitude)  distribution  in  the  wavenumbers  has  a 
bearing  on  the  predictability  of  a  forecast,  such  that  when 
the  long  waves  were  stationary  the  medium-range  forecast  was 
good.  However,  when  the  long  waves  were  in  transition,  the 
medium- range    forecast    was    poor. 

This  technigue  can  also  indicate  the  flow  characteris- 
tics of  a  height  field.  Meridional  flow  is  related  to  high 
power  (amplitude)  in  the  lower  wavenumbers  (waves  1-3), 
while  zonal  flow  is  more  a  function  of  the  power  in  the 
higher    wavenumbers    (4-6) . 

To  investigate  how  the  zonal  wavenumber  structure  of  the 
height  fields  is  related  to  medium-range  predictability,  the 
Fourier  decomposition  method  is  used  to  determine  the  waven- 
umber versus  amplitude  spectrum  of  each  500  rab  analysis 
height    field-  The   standard    fast    Fourier      transform   method 

was  employed  (II-ISL,  1982).  The  500  mb  analysis  field  height 
anomalies  in  Figs.  A, 14  and  A. 15  were  used  as  the  input 
waveform  for  the  Fourier  decomposition  routine.  The  ampli- 
tudes (in  meters)  for  each  day  are  plotted  as  a  function  of 
the    wavenumber   in   Figs.    A. 20     (winter    83-84)     and    A. 21     (winter 
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84-85).  As  in  the  previous  Hovmiilier  diagrams,  time  is 
increasing  towards  the  top- 
No  one  "signature"  of  a  denoted  STAEH  or  FLOP  period  is 
common  to  both  winters  in  this  harmonic  analysis.  For  the 
1983-84  winter  season,  the  most  characteristic  feature  of 
the  STAEHs  is  the  peak  in  wave  number  three  (often  associ- 
ated with  a  blocking  situation  as  in  Fig-  A.  9)  and  rela- 
tively lower  amplitudes  in  waves  two  and  four.  However, 
this  chciracterization  of  the  wavenumber  structure  of  STAEHs 
and  FLOPS  in  the  83-84  winter  is  not  unique  to  these  periods 
alone.  For  example.  Fig.  A.  20  shows  that  there  are  peaks  in 
wave  three  on  7  and  17  January  1984  and  on  9  February,  yet 
these  are  not  STAEHs.  Similarly,  20  December  1983  and  14 
February  have  distinct  minimums  in  wave  three,  yet  lie 
outside  of  the  FLOP  periods- 
There  is  no  correspondingly  high  amplitude  in  wave 
number  three  for  all  the  STASH  periods  of  winter  84-85  (Fig. 
A. 21).  No  conclusion  can  be  drawn  about  a  characteristic 
structure  that  would  indicate  a  STAEH  or  a  FLOP  period  since 
the  data  set  is  too  small  considering  the  variances  encoun- 
tered   within    the    wavenumber    amplitudes. 

The  indication,  discussed  earlier,  of  STAEHs  to  exist 
when  the  flow  pattern  shifts  from  a  meridional  to  a  zonal 
pattern  can  also  be  detected  as  a  shift  in  the  amplitude  of 
the  zonal  wavenumbers  from  the  analysis  amplitude  spectrum 
to   that    of    the    verifying    amplitude    spectrum- 

To  facilitate  detecting  these  shifts  in  the  amplitude  of 
the  major  wavenumbers  for  both  winters,  a  Hovmiilier  diagram 
has  been  constructed  that  shows  the  change  in  amplitude  over 
the      five-day   forecast      period  for      each    wavenumber-  Fig. 

A-22  graphs  the  five-day  tendency  of  the  wavenumbers  in 
winter  1983-84,  and  Fig-  A. 23  is  the  same  for  winter 
1984-85-  The  tendencies  are  calculated  by  subtracting,  for 
each    wavenumber,       the    day    one   value    from   the    day    five    value- 
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Positive  values  reflect  growth  in  a  wavenumber,  and  negative 
values  reflect  decay.  The  convention  for  these  diagrams  is 
that  the  difference  is  plotted  as  a  function  of  day  one  and 
(as  before)  time  increases  as  one  moves  toward  the  top  of 
the  diagram- 
It  is  expected  that  a  STARH  period  will  experience  a 
decrease  in  amplitude  in  the  longer  waves  and  possibly  an 
increase  in  power  in  the  higher  wavenumbers.  A  shift  in 
amplitude  in  this  sense  would  be  an  indication  that  the  flow 
pattern  is  shifting  from  a  low  index  (meridional  flow)  to  a 
high  index  (zonal  flow)  regime,  FLOPS  would  be  expected  to 
reflect  just  the  opposite  behavior. 

This  analysis  technique  meets  with  more  success  than  did 
the  standard  deviation  differences  technique,  as  more 
periods  are  explained  by  this  method.  Table  2  lists  seven 
of  13  periods  that  have  shifts  in  wavenumber  amplitudes  as 
expected  (S2,  F^,  F4,  S11,  S12,  F11  and  F12),  while  the 
standard  deviation  method  can  only  account  for  four  of  the 
13  periods.  On  the  other  hand,  period  F8  shows  a  tendency 
that  is  just  the  opposite  to  what  is  expected.  This  period 
also  showed  a  similar  result  for  the  standard  deviation 
method.  The  remaining  five  periods  show  show  some  of  the 
expected  shifts  in  zonal  wavenumber  amplitude,  but  also  some 
complex  behavior  not  anticipated.  For  example,  in  Fig.  A. 22, 
STARH  SI  does  show  a  decrease  in  amplitude  at  the  beginning 
of  the  period  (26  December)  in  wave  two,  but  wave  one 
increases  though  most  of  the  period.  On  27  December  wave 
three  shows  decay,  and  wave  five  growth  (as  expected)  ,  but  a 
shift  occurs  on  29  December  when  wave  one  begins  to  decay 
and  wave  two  shows  growth.  Waves  four  and  five  (as 
expected)  show  growth  from  then  until  the  end  of  the  period. 
Other  periods  like  this  are  S5  and  F7,  In  the  second 
winter  (Fig.  A. 23)  the  periods  showing  expected  shifts  in 
wavenumber  amplitudes  are  S11,  S12,  and  F11.  Those  showing 
more  complex  behavior  are  SI  0,  FIO  and  F12. 
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Unfortunately,  the  data  set  for  these  two  winters  is  too 
small  to  be  able  to  statistically  show  that  one  pattern  of 
behavior  in  the  complex  cases  is  significant,  but  it  is 
clear  that  many  periods  are  explained  very  well  by  this 
technique.  Most  interesting  is  period  F8  which  (as  it  did 
in  the  standard  deviation  difference  results)  shows  results 
just  the  opposite  to  what  was  expected  (i.e.,  the  long  waves 
gained  power,  while  wave  four  and  six  lost  power  over  the 
forecast  period).  Also    intriguing      are  those      periods   not 

scored  hign  or  low  with  the  ACZ ,  yet  exhibit  tha  same 
tendencies  as  a  STARH  or  a  FLOP  (such  as  15-17  December 
1984:  FLOP-liJce  shift  in  the  wavenumbers,  and  15-17  January 
1984:  STARH-like  wavenumber  shift).  This  means  that  the 
results  so  far,  though  promising,  are  still  not  unique  to 
the    STARHs    and    FLOPs. 

F-       MEBIDIONAI    WAVENUMBER    STROCTORE 

The  meridional  structure  of  the  flow  patterns  may  be  an 
important  factor  in  determining  model  performance. 
Investigation  of  the  north-south  extent  of  the  various  zonal 
wavenumbers  is  the  subject  of  this  final  section- 
It  is  possible  to  depict  the  meridional  structure  of  the 
flow  pattern  of  an  analysis  field  by  examining  the  meri- 
dional wavenumber  spectrum.  This  spectrum  can  be  calculated 
by  using  a  spherical  harmonics  decomposition  of  the  500  mb 
height    field. 

The  spherical  harmonics  decomposition  of  the  wave  field 
depicts  the  amplitude  of  wavenumbers  in  two  dimensions, 
zonal  and  meridional.  In  this  study  the  spherical  harmonics 
decomposition  was  conducted  using  a  triangular  truncation  at 
N=33       ("N"       being      the         zonal      wavenumber)  (Haltiner      and 

Williams,    1980)  . 

The  wave  numbers  in  the  meridional  direction  are  defined 
in  this  study  as  "pole  to  pole"  wavenumbers-  In  plotting, 
the    amplitude    values    (in    meters)       for    the   pole    to    pole    waves 
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with  the  same  number  of  nodes  (the  same  |M-N|  wavenumber 
values,  where  M  is  the  zonal  wave  number),  were  summed  and 
plotted   as   a    function    of    pole    to   pole   wavenumber- 

Several  plots  of  the  pole  to  pole  wavenumber  structure 
did   not      produce  any      significant   results.  It   was      noted, 

however,  in  studying  the  zonal  wavenumber  structure  of  the 
1983-84  winter  season  (Fig.  A, 20)  that  one  difference 
between  the  STARH  and  FLOP  periods  was  a  peaJc  in  zonal  wave 
three  in  the  STAEHs,  and  a  minimum  amplitude  value  in  zonal 
wave  three  for  the  FLOPS.  To  examine  the  meridional  extent 
of  this  wave,  the  analysis  height  fields  of  all  the  days 
that  compose  each  individual  STARH  and  FLOP  period  were 
averaged  together  to  form  a  representative  height  field  of 
that  period.  This  height  field  was  used  as  the  input  height 
field  for  the  spherical  harmonics  decomposition-  Fig.  A. 24 
shows  the  meridional  wavenumber  structure  for  zonal  wave 
three  of  all  the  STARH  periods  of  winter  83-84  ani  Fig- 
A.25  shows  the  same  for  the  FLOPs  in  the  1983-84  winter 
season. 

To  interpret  these  diagrams  one  must  recall  that  a 
higher  amplitude  in  the  lower  pole  to  pole  wavenumbers 
(|M-N|  nodes)  would  indicate  less  meridional  structure,  or  a 
more  meridional  flow  component.  If  the  premise  is  correct 
that  good  five-day  forecasts  are  generated  on  days  when  the 
flow  patterns  are  more  meridional  (low  index  situation)  and 
the  poor  forecasts  are  generated  on  days  when  the  flow 
patterns  are  more  zonal  (high  index  situation),  then  it  is 
expected  that  the  STARHs  would  saow  more  amplitude  in  the 
lower  pole  to  pole  wavenumbers,  while  the  FLOPs  would  be 
expected  to  show  more  amplitude  in  the  higher  pole  to  pole 
wavenumbers. 

In  contrasting  the  amplitude  in  pole  to  pole  wavenumbers 
one  and  four  between  Figs.  A. 24  and  A. 25,  the  STARH  periods 
have  a    higher      amplitude     (more  power)       in   wave      one    than    the 
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FLOPS.  Also,  the  FLOPS  show  more  amplitude  (power)  in  wave 
four  that  do  the  STARHs.  A  close  inspection  of  Fig.  A. 25 
shows  F8  is  an  extreme  case  of  having  more  power  in  the 
higher  pole  to  pole  wavenumbers-  Recall  that  F8  provided 
unusual  results  using  the  other  techniques  previously 
discussed    in    this  chapter. 

In  summary,  all  of  the  STARH  and  FLOP  periods  were 
accounted  for  in  the  winter  season  1983-84  by  this  tech- 
nique. Table  2  shows  this  result  for  the  STARHs  and  FLOPs 
of  winter  1983-84  is  quite  dramatic.  This  technique  works 
well  in  delineating  the  good  from  the  poor  medium-range 
forecasts   in    this   winter. 

This  same  approach,  however,  does  not  work  well  for  the 
STARHs  and  FLCPs  in  winter  1984-85.  Figs-  A. 26  and  A.  27 
illustrate  the  meridional  wavenumber  structure  of  zonal  wave 
three      for    the      STARHs   and      FLOPs   of      winter    1984-85.  The 

construction  and  interpretation  of  these  figures  is  the  same 
as  for  those  of  the  previous  winter  (Figs.  A. 24  and  A-25). 
By  contrasting  the  STARHs  on  Fig.  A. 26  to  the  FLOPs  on  Fig, 
A. 27,  one  can  detect  a  slightly  lower  amplitude  in  pole  to 
pole  wavenumbers  three  and  four  of  the  STARHs  as  compared  to 
the  FLOPS.  This  is  certainly  expected,  but  overall,  there 
is  not  the  strong  difference  between  the  periods  as  in  the 
previous  winter.  F11  and  S11,  for  example,  are  not  very 
different  from  each  other,  and  the  peak  in  wave  two  of  S12 
is   more    suggestive    of    what   is   expected    from   a    FLOP   period- 

The  most  interesting  feature  of  Fig.  A.  26  is  the  period 
S10.  The  exceptionally  large  peak  in  wavenumber  one  is 
typical  of  the  other  STARHs.  It  should  be  noted  that  for 
this  analysis  technique,  only  the  last  half  of  the  days 
composing  S10  were  included  in  the  averaging  process  that 
produced  the  average  height  field  to  represent  S10  (i.e., 
days  26-31  December  were  left  out).  This  was  done  since  S1G 
was    a      very    long      period     (12       days    long)         and    since      it    has 
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already  been  seen  that  some  other  process  was  occurring  at 
the  beginning  of  this  period  to  cause  the  ACC  score  to  be 
large.  By  isolating  the  first  half  of  the  period  it  makes 
more  apparent  what  is  happening  in  the  last  half  of  the 
period  S10  to  cause  the  high  ACC  score-  Computation  of  the 
meridional  wavenumber  structure  of  the  entire  period, 
however,  showed  the  graph  of  S10  t.o  resemble  those  of  S11 
and   S12. 

In  general,  though,  the  meridional  structure  of  zonal 
wave  three  is  not  as  successful  in  the  winter  1984-85 
season-  It  is  apparent  from  the  wavenumber  structure  of  tne 
winter  season  1984-85  on  Fig.  A,  21  and  for  winter  1983-84  on 
Fig-  A. 20  that  these  two  winters  definitely  had  a  different 
character.  This     is    emphasised      by   the      strong    wave      three 

structure   of    the   STAEHs  in   1983-84,    in    agreement    with   Quiroz 
(1984)     on    the    description    of    the    19  83-84    winter    season- 

Quiroz  noted  tnat  the  1983-84  winter  was  a  season  of 
strong  blocking  and  of  intensely  cold  surface  temperatures 
for  North  America  (setting  several  low  temperature  records 
and  qualifying  as  one  of  the  six  coldest  winters  on  record) - 
A  quick  check  of  records 'of  the  19  84-85  winter  reveals  that 
it  was  not  nearly  so  spectacular  a  season  as  the  previous 
winter. 

Since  these  two  winters  show  a  marked  difference  in 
character,  other  combinations  of  wavenumbers  were  examined 
for  the  winter  season  1984-85  that  might  show  a  difference 
in  character  between  the  STARHs  and  FLOPs-  Extensive  exper- 
imenting and  examining  the  zonal  wavenumber  structure  of  the 
1984-85  winter  season  indicated  a  difference  in  the  STARHs 
and  FLOPS  in  the  zonal  wavenumbers  four  through  six- 
Figs-  A-28  and  A. 29  show  the  meridional  wave  structure 
of  zonal  waves  four  through  six  for  the  STARH  and  FLOi? 
periods  of  the  1984-85  winter  season.  These  graphs  differ 
from   Figs.       A, 24   and    A. 25    only  in    that    the    meridional   struc- 
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ture  of  the  combination  of  zonal  waves  four  through  six  is 
depicted. 

These  two  figures  (A. 28  and  A. 29)  show  a  clear  differ- 
ence in  the  shape  of  the  curves  for  the  STARHs  and  the 
FLOPS.  As  expected,  the  STARHs  have  a  high  peak  at  pole  to 
pole  wavenumber  one.  In  general,  the  FLOPs  have  a  high  wave 
one  amplitude  as  well,  but  .at  wave  four  they  clearly  have  a 
larger    amplitude   than    do    the    STARHs.  F10    does   not    have   as 

high  an  amplitude  at  wave  four  '  as  do  the  other  two  FLOPs, 
but    it    also   has   a  correspondingly    lower   wave   one    amplitude. 

As  seen  in  Table  2,  this  technique  correctly  indicates 
the  expected  structure  of  all  the  STARHs  and  FLOPs  in  tae 
winter  season  1984-85.  However,  examining  the  structure  of 
the  meridional  wavenumbers  for  zonal  waves  four  through  six 
does  not  provide  any  differentiation  between  the  STARHs  and 
FLOPS   of  the    previous    winters. 

G.       SUMMARY    OF    ANALYSIS    TECHNIQaES 

Table  2  lists  for  all  the  periods  selected  in  both 
winters  the  ability  of  the  various  analysis  techniques  used 
in  this  chapter  to  describe  the  features  expected  to  be 
found    in      STARHs   and      FLOPs.  A      rather   simplistic      grading 

scheme  is  employed  to  summarize  the  results  of  all  the  tech- 
niques. An  "X"  listed  in  the  row  next  to  a  period  indicates 
that  the  analysis  technique  listed  in  the  column  above  the 
"X"  showed  expected  results.  A  "/"  (slash)  denotes  some  of 
the  days  of  the  period  showed  expected  results  or  that  sojie 
of  what  was  expected  to  be  present  was  present,  but  not  in  a 
spectacular  or  a  unique  fashion-  A  "-"  (dash)  indicates  no 
clear  indication  was  observed,  and  an  "0"  is  used  when 
results  were  obtained  that  were  the  opposite  of  what  was 
expected. 

If  one  analysis  technique  was  to  be  a  perfect  indicator 
of  STARHs  and  FLOFs,  one  would  expect  to  find  nothing  but 
"X' s"    in   the    column   beneath    it.      No    one   technique    is   perfect 
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for  the  STARHs  and  FLOPs  of  both  winters,  but  it  is  inter- 
esting to  note  that  between  both  the  standard  deviation 
differences  and  the  Fourier  decomposition  differences 
methods  only  three  periods  are  aot  fully  "as  expected". 
Also  interesting  is  that  the  Spherical  Harmonics  method  can 
explain  all  the  periods  of  one  winter  if  the  correct  zonal 
wave  or  waves  are  used-  Persistence,  as  discussed  earlier, 
is  not  a  consistent  indicator  of  STARHs  and  FLOPs.  This  can 
be  seen  as  almost  an  equal  mix  of  "X*s"  and  "O's"  appear 
under  that  column.  Early  error  growth  is  also  discounted  as 
a  means  to  detect  model  performance  as  under  the  "Early 
Errors"  column  dash  symbols  (indicating  no  clear  indication 
observed)  appear  for  all  periods  except  F8.  This  indicates 
that  only  in  rare  occasions  does  this  method  apply. 
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V.    CONCLO SIGNS    AND    BECOBH ENDATIONS    FOB    FOTWRE    RESEARCH 

The  main  conclusion  of  this  stady  is  that  it  is  possible 
to  quantify  certain  aspects  of  flow  patterns  that  are  indi- 
cative of  STARH  and  FLOP  periods.  These  indications 
include : 

•  Long  wave  amplitude  decay  over  the  forecast  period  can 
lead   to      a    good      forecast,      while      long   wave      amplitude 

?rowth    over   the      forecast   period    tends    to      produce    poor 
orecasts. 

•  Transition  of  flow  pattern  from  a  meridional  (low 
index)  to  zonal  (higii  index)  regime  leads  to  a  good 
forecast  period-  The  opposite  case  leads  to  a  poor 
forecast    period. 

•  STARHs  are  related  to  periods  with  less  meridional 
structure  (more  power  m  the  low  meridional  wave 
numbers)  while  FLOPS  are  related  to  periods  with  more 
meridional    structure. 

None  of  these  indications,  however,  show  a  unique  or  over- 
whelming result.  Periods  do  exist  for  each  of  these  indica- 
tors that  fit  the  criterion,  but  do  not  score  overwhelmingly 
high   or    low    using   the    ACC.  Also,       some   periods    that    score 

exceptionally  high  or  low  have  flow  indications  (as  listed 
above)  that  are  just  opposite  to  what  was  expected  (i.e.,  a 
STARH  had  flow  characteristics  indicative  of  a  FLOP  and  vice 
versa)  . 

There  is  more  confidence  placed  in  the  conclusions  that 
the    variance    in    the    ACC   was    not   due    to    the   following: 

•  Persistence   as   a   good  indicator   of    model    perf ormance. 

•  Early  error  growth  in  FLOP  periods  due  to  low  data 
availability   or   some   critical    data   missing. 

•  Errors  consistently  occurring  in  the  same  geographical 
area   causing    FLOPs. 

Additionally,    the    data   base   available   for   this    study   was 
inadequate    to    provide    a      statistically    significant   basis   for 
determining    confidence    levels    of    the    listed   indications- 
Based    on    the   findings   of    this    study,    several    recommended 
future    research    efforts    are    delineated      here    that    will    hope- 


38 


fully  further  our  understanding  of  high  and  low  predict- 
ability periods  in  medium  range  forecasts- 
First,  it  is  recommended  to  investigate  the  STARH  and 
FLOP  periods  in  more  detail  to  determine  why  certain  anal- 
ysis methods  used  in  this  study  were  successful  in 
explaining  the  existence  of  some  high  and  low  predictability 
periods  but  not  for  others, 

A  second  recommendation  is  to  continue  computing  ACCs  of 
NOGAPS  5  day  forecasts  to  build  up  a  more  significant  data 
base  for  statistical  verification  purposes.  Expanding  the 
seasons  examined  will  also  add  to  the  statistical  data  base, 
but  will  also  indicate  wether  or  not  the  mechanisms  that 
cause  STARHs  and  FLOPs  in  these  other  seasons  are  ths  sane 
as  the  ones  for  the  winter  seasons. 

Third,  it  is  recommended  to  calculate  the  ACC  for  the 
five-day  forecasts  at  additional  levels  in  the  atmosphere  to 
allow  other  flow  patterns  to  be  investigated.  The  final 
recommendation  is  to  investigate  the  possibility  that  some 
combination  of  the  parameters  measured  in  these  analysis 
techniques  might  provide  an  indication  of  the  potential 
predictability  of  a  medium  range  forecast. 
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Fig-    A. 3         ACC   versus   forecast   day    for   winter    ISSJ-S** 
STARH    periods.      Each    ACC    value    plotted    represents   an 
average   value    for   the   period. 
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Fig-    A. 4         ACC   versus   forecast   day    for   winter    1983-84 

FLOP  periods.      Each    ACC    value   plotted   represents   an 

average   value    for   the   period. 
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Fig.    A. 5        ACC   versus    forecast   day    for   winter    1984-85 
STAEH   periods.      Each    ACC    value   plotted   represents  an 
average   value    for    the   period- 
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Fig-    A. 6        ACC   versus    forecast   day    for   winter    1981-85 

FLOP  periods.      Each    ACC    value   plotted   represents   an 

average   value    for    the   period. 
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DEC  1983 
500MB  TAU 
23  DEC  83 

CONTOUR  =  e.ooon 

WAVE  GROUP  =  0 


Fig-   A- 7        23   Deceaber   1983   500    mb    height   analysis. 
Northern    Hemisphere   Polar   Stereogr aphic   projection. 
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500MB  TAU 
23  DEC  ERROR 
CONTOUR  =  160.0 
WAVE  GROUP  =  0 


Fig.    A. 8         Error   patterns    for    23    December    1983    500    mb 
five-day    forecast    field  minus   2  8   December    1983 
500   ab   analysis   field. 
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DEC  1983 
5()0MB   TAU 

28  DEC  83 
CONTOUR  =  6.000*1 
WAVE  GROUP  =  0 


Fig.    A. 9         28    December    1983    500    ab    height   analysis. 
Northern    Hemisphere   Polar   Stereographic    projection. 
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500MB  TAU 
28  DEC  ERROR 
CONTOUR  =  160.0 
WAVE  GROUP  =  0 

Fig«    A- 10         Error   patterns   for   28   Deceaber    1983   500    mb 
five-day    forecast    field  minus   02  January    1984 
500    ab   analysis    field. 
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JAN  19B4 
500MB  TAUO 

02  JAN  84 
CONTOUR  =  6.000* 
WAVE  GROUP  =  0 


Fig.    A. 1  1         02    January    1984    500    mb    height    analysis. 
Northern    Hemisphere  Polar   Stereographic    projection. 
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Fig.    A.  12        Winter    1983-84   day-five   error   patterns 

(average    five-day   forecast    500    nb    heights    minus   average 

verifying  day  an   heights). 
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Fig.    A.  13        Winter    1984-85   day-five   error    patterns 
(average    five-day    forecast    500    mb    heights   ainus   average 

verifying  day  an  heights) . 
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Fig.  A. 20    Winter  198  3-84.   Amplitude  of  zonal 
Mavenumbers  of  average  daily  500  ab  height  fields  from 

Figure  A- 14. 
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Fig.    A. 21         Winter    1984-85.      Amplitude   of    zonal 
wavenumbers   of    average   daily   500    mb    height    fields   froi 

Figure  A- 15. 
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^AVENUMDER 

Fig.    A. 22         Sinter    1983-84.       Five-day    amplitude    tendencies 
of    zonal    uavenumbers,      Day-5    minus    day-1    differences 
plotted   as  a    function    of    day   one. 
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Fig.    A-23         Winter    ISStt-SS-       Five-day    amplitude    tendencies 
of    zonal    wavenumbers.      Day— 5   minus    day- 1    differences 
plotted   as   a    function   of    day   one. 
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APP EHDIX    B 
TABLES 


TABLE    1 
SDHflARy   OF   SELECTED    PERIODS 


Winter 

Period 

Days   when    5 
day   forecasts 
were   generated 

Average   ACC 
of    period 

83-84 

SI 

26-30 

Dec 

83 

0.705 

S2 

19-21 

Jan 

84 

0,699 

S5 

08-09 

Dec 

83 

0.642 

Avg-  = 
S.D-  = 

0. 
0. 

500 
134 

F1 

30-31 

Jan 

84 

0-301 

F4 

22-23 

Dec 

83 

0.295 

F7 

04-06 

Feb 

84 

0.250 

F8 

16-21 

Feb 

84 

0.  267 

84-85 

S10 

2  7    Jan-0  7 
1985 

Feb 

0-713 

S11 

27-31 

Dec 

84 

0-675 

S12 

14-15 

Ja  n 

85 

0.678 

Avg.= 
S.  D.= 

0. 

0. 

490 
171 

F10 

08-12 

Dec 

64 

0.  160 

F11 

15-20 

Feb 

85 

0.281 

F12 

02-04 

Jan 

85 

0.  288 
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TABLE    2 
SOHMABY    OF    AHILTSIS    RESULTS 


/    o    / 


SPHERICAL 
HARMONICS 


/ 


/ 


/ 


WINTER 

PERIOD 

/  ^^ 

/ 

/  <b"  <d"  o 

/  -v  ^^  o 

/  -V"^''  'b 

V 

83-8* 

51 

X 

——_—_—• 

X 

/ 

X 

- 

52 

0 

/ 

X 

—  ^  ■""•  —  "  *" 

X 

- 

S5 

0 

- 

X 

/ 

^ 

- 

F  I 

c 

- 

/ 

X 

X 

- 

F  4 

X 

- 

/ 

X 

X 

- 

PT 

X 

- 

/ 

/ 

X 

- 

F5 

0 

X 

0 

Q 

X 

- 

8  a  -  b  5 



r.  1  0 

/ 

- 

/ 

/ 

- 

X 

S  I  I 

X 

- 

/ 

X 

- 

X 

?!  2 

--^ 

- 

X 

X 

_ 

- 

X 

'-'  I  0 

- 

X 



- 

X 

-  I  1 

0 

- 

/ 

-._  —  _  —  _  — 

X 

- 

X 

F  12 

X 

- 

/ 



— 

_  _ 

- 

X 



•  X  •  = 

.  /  •  = 

•  —  •  = 

•  0  •  = 


LEGJND 

Expected  result 

Marginal  result 

No  significant  result 

Results  Opposite  to  expectation 


70 


TABLE    3 
HINTEB    83-84    ACC    STATISTICAL    SaMMARI 


Pa  ram e  ter 

Win  ter 
Summary 

Dec- 

Jan. 

Feb, 

Average    Forecast   ACC 

0.500 

.53 

.53 

.44 

Standard   Deviation 

0-  132 

.13 

-13 

.  15 

Correlation    of 
fcst.    and   per. 

0.  155 

+  .47    , 

-.30 

—         -        « 

-,20 

Number    data    points 

87 

31       , 

3T 

29 

TABLE    4 
WINTER    84-85    ACC     STATISTICAL    SOMMARY 


Winter 
Parameter                                      Summary 

Dec. 

Jan. 

Feb- 

Average   Forecast   ACC                  0-490 

,46 
.18 

,50       1 

.52 

Standard  Deviation                       0.171 

,16       1 

.  17 

Correlation    of                                0.138 
fcst-    and   per. 

+  .26    i 
1 

+  .24 

-.  11 

Number    data    points                       8  2 

31       1 

31       1 

23 
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